Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add DuckDB EmbeddingStore implementation (#2183) #24

Merged
merged 3 commits into from
Nov 29, 2024

Conversation

fb64
Copy link
Contributor

@fb64 fb64 commented Nov 27, 2024

Issue

Closes langchain4j#2183

Change

Add implementation of EmbeddingStore with DuckDB that supports metadata.
Replace langchain4j/langchain4j#2184

General checklist

Checklist for adding new maven module

  • I have added my new module in the root pom.xml and langchain4j-bom/pom.xml

Checklist for adding new embedding store integration

  • I have added a {NameOfIntegration}EmbeddingStoreIT that extends from either EmbeddingStoreIT or EmbeddingStoreWithFilteringIT
  • I have added a {NameOfIntegration}EmbeddingStoreRemovalIT that extends from EmbeddingStoreWithRemovalIT

@Martin7-1 Martin7-1 added P2 High priority P3 Medium priority theme: embedding store Issues/PRs related to embedding store labels Nov 27, 2024
Copy link
Collaborator

@Martin7-1 Martin7-1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fb64 Thank you! Could you please check tests? Looks like it passed in my local env but failed in Github Actions.

@fb64 fb64 force-pushed the duckdb branch 2 times, most recently from bf64ccd to 18a035b Compare November 27, 2024 14:14
@fb64
Copy link
Contributor Author

fb64 commented Nov 27, 2024

@fb64 Thank you! Could you please check tests? Looks like it passed in my local env but failed in Github Actions.

I'll check, but actually everything passed on my local env too 🤔

@fb64
Copy link
Contributor Author

fb64 commented Nov 27, 2024

I think I fixed the issue, by casting the sql string into float[].
However I don't know why it was working on MacOs and not on linux, probably a matter of float precision 😓 ....
bf47e05#diff-17af6f43ebda2eec7140d121bb2b7aa3df64d1a3de0e261eebc473bda2080dbeR289

Copy link
Collaborator

@Martin7-1 Martin7-1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fb64 Thank you! Could you please add docs in langchain4j main repo?

@Martin7-1 Martin7-1 merged commit 0cb6ea4 into langchain4j:main Nov 29, 2024
4 checks passed
@fb64 fb64 deleted the duckdb branch November 29, 2024 06:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 High priority P3 Medium priority theme: embedding store Issues/PRs related to embedding store
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Add support to use DuckDB as EmbeddingStore
2 participants